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EVALUATION 


Evaluation keeps an organization on course and prevents it from flying blind. 
Evaluation is simply the process of finding honest and objective answers for 
questions like: How well are we doing what we are doing? Evaluation can be 
used to judge or measure attitudes and ideas, programs, projects and even 
goals. These judgements have to be based on objective evidence and they have 
to be made systematically. 


STEPS IN EVALUATION 


Suppose a community group is planning to raise funds to purchase an enclyco- 
pedia for the school library. The evaluation process would work like this: 


1. What is the target ? 
eee en 


The committee with the job of fund raising will probably first 
survey the need and decide which enclycopedia to buy. If the 
group is convinced, on the basis of facts, workers available, 
and other factors involved, then it can raise the $1000 needed 
to buy the fine encyclopedia the experts have recommended 
they have set the goal. 


2. Where are we now? 
ee ee a 2 OW. | 


Once they have the goal, they have a standard against which to 
measure their progress. It will be important to know what 
results their methods are producing. A careful record of 
progress toward the objective will be useful in future planning. 


3.2 What activities will lead toward the goal? 
sa ee ee OWA LO Ne 60a 


The committee must now decide what form their fund raising 
will take. They decide on a Christmas bazaar because they 
feel it will attract many people in the community. The 
committee begins the search for volunteers to provide the 
items for sale. After a week, the committee members 
report little success. An analysis of information reveals 
that the Christmas bazaar will be poorly attended because 
other community affairs are planned for the same week. 

On the basis of this evidence the committee now changes 
its course of action. It decides to plan for a community 
party after Christmas — a Valentine party in fact. The 
available facilities are surveyed, a program plan drawn 
up, publicity arranged and tickets distributed for sale. 


4, How are we doing ? 


Here is where records of the effectiveness of various methods 
used in the past will be useful. Suppose, after ten days the 
ticket sale is disappointing compared with the committee's goal 
and expectations. The members will have to look for possible 
reasons. 


5. What does the collected evidence mean ? 


A study of the record of the ticket sale, comparisons with past 
sales, and other evidence may show that few people are talking 
about the project or that some large group, such as teenagers 
are not interested. 


6. Are we doing what we should be doing ? 


More canvassers, a telephone "blitz" and posters are arranged 
to give more publicity. The program of the party itself might 
be changed to include features to attract teenagers or other 
groups who have not been buying tickets. 


7. Can we reach the objective ? 


If it becomes clear that not any of the committee's efforts can 
possibly produce the needed $1000, the goal for the project may 
have to be cut down, perhaps to half. The committee would then 
begin to plan another project to be carried out later. The 
objective for the new project would be the other $500 needed 

to buy the encyclopedia. The committee might decide to modify 
its objective in other ways. For instance cut it by one-third 
and buy a less expensive encyclopedia. 


ANOTHER APPLICATION OF THE STEPS 


For another example, suppose that at a Home and School Association meeting 
in November some people are interested in setting up an adult education 
program. Evaluation will play an important role in the project. 


Before members decide their goals, they should try to find out the specific 
adult education needs in the community. They should consider the local 
facilities and whether leadership is available. They should decide how many 
classes will be held and when. 


Eventually, they might offer three courses at classes held from 7:30 to 9:30 
on Wednesday evenings, beginning the second Wednesday in January and 
ending the third Wednesday in April (15 weeks) with a minimum enrolment 
of 15 people in each class. 


Having set their goal, the committee can measure their progress. Allowing 
for the Christmas rush, they have about six weeks to get the project under way. 
They might appoint a committee to meet once a week to follow up registrations, 
arrange for staff and class-rooms and take care of routine details. 


Now, the committee may notice early in December that there is little enthusiasm 
for one of the courses. Perhaps only four or five persistent talkers had 
originally wanted it. Meanwhile, the committee has seen that another course 
will require three or four leaders to take care of the numbers registering for it. 


The committee must now find solutions to two problems. It can make a closer 
canvass for participants in the unpopular course, substitute a more appealing 
subject, or limit the project to the two popular courses. The Association may 
go outside the community to find enough leaders for the over-subscribed course 
or they may adopt a study-group approach that requires no lecturers. 


The committee may face decisions of this nature up to the week before the 
courses begin. As new evidence turns up, the committee will interpret it 
and modify decisions and actions in the light of new facts. Some Association 
members may feel they are settling for less than the ideal goals originally. 
However, only frustration will result from continuing to strive for objectives 
that are unobtainable on the basis of good, sound evidence. 


Once the classes start, a new evaluation cycle begins. Each instructor may 
apply the seven steps in evaluation to find out how well the classes are working 
toward the goal. The evidence they analyze and interpret is made up of the 
opinions expressed by members of the classes. When the classes are 
completed in April, the committee will do an evaluation of the entire course, 
methods of presentation, times chosen, facilities and suggestions for next 
year. In short, good evaluation is a continuous process. 


PRINCIPLES OF EVALUATION 


Good evaluation meets people on their own ground. The ways people think 
and the interest they take in any subject will vary with their education and 
background. Standards used in evaluation should be those that will put the 
next improving step within easy reach of the group. It would be discouraging 
to apply the standards of the London or Broadway stage to the productions of 
an amateur dramatic group at a local church. 


The best evaluation is co-operative. Sharing the work of measuring progress 
leads all members toward deeper understanding and greater desire to improve 
than if some higher authority carries out the evaluation for them. 


Useful evaluation considers every angle. Evaluators should consider every- 
thing related to the activities or undertaking being evaluated. The key factor 
may turn out to be one of the least obvious and something no one had 
considered. ; 


Business-like evaluation is continuous. No community organization or committee 
should wait until failure looms before they apply systematic evaluation. 
Different techniques may be used at different stages, but evaluation should 


be always going on. 


Effective evaluation uses frank and probing questions. Evaluation must 
criticize and diagnose not just activities, but also the ultimate goals. Goals 
we set for ourselves are not sacred. They should be reviewed periodically in 
the light of new experience and information. The purpose of evaluation is 
more than just finding out whether a desired result is or is not being achieved. 
Its aim is also to find out why a given course of action does or does not 
produce the desired result. 


All evaluation leads to action. The only purpose of evaluation is to improve 
programs and activities aimed at goals and to improve the goals themselves 
in the light of individual and social needs. 


WHO EVALUATES ? 


Because participation in community organizations is voluntary, it is good to 
think of evaluation as a group activity. Members should evaluate their own 
actions. No outsider should decide whether they have succeeded or failed. 
Outside experts are useful mainly in helping to devise methods for collecting 
the required kind of information and in helping the group to analyze and 
interpret that information. The final judgment or evaluation should be made 
by the group. 


WHAT IS EVALUATED? 


In a community organization, evaluation is often concerned with the organiza- 
tion itself, or the effects of the organization's work on its members and on 
the community. 


The organization needs to know how well it functions. What proportion of 
members attend meetings regularly? How well do members participate in 
activities? Is the organization making full and good use of all its resources 
of leadership and finances? How effective are its committees ? 


Evaluation of activities in terms of their effect on members will help keep 
the programs in tune with members' needs. Are there changes in the 
knowledge of members or in their behavior as members, neighbors and 
citizens? Are they growing more mature and objective about their own 
and others behavior? Harmony, or lack of it, among the members, and 
the effectiveness of the methods being used to deal with tensions are other 
factors that may be tested. 


The impact of the organization on the community may also be measured. 
How far is the organization helping to meet community needs? Does it have 
an appreciable effect on the co-operation among groups ? 
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EVALUATION AND GOALS 


Before they can determine the effectiveness of meetings, programs or activities 
members must have clear targets or goals. If these goals are too general, , 
evaluation will be difficult. For instance, a group that plans a series of 
meetings "to promote good citizenship" will find it hard to measure any - 
progress. But a series of meetings planned "to introduce new Canadians to 
community" has a well-defined target. 


In setting goals there are several questions to bear in mind. 


m@ Are the goals in line with aims, objects and policies of 
the organization ? 


m Are they acceptable to the members concerned? Are they 
expressed in words members really understand? 


m Are they clear enough to be practical ? 


w Is the idea contained in the goal in line with the way most 
members think? Does it set too high or too low a standard 
for this group ? 


# Are the goals attainable? Can they be measured ? 


EVALUATION TECHNIQUES 


There are no hard and fast rules in evaluation. The measurement of progress 
is simple enough if all we have to do is collect statistics — for example, 


a number of people who buy tickets for a dance or enrol ina 
workshop, 

ms number of people who respond to a fund-raising campaign, 

gw totals of donations by days or weeks, 


ms number of votes for and against questions submitted to members. 


But more often than not, community groups want to find out things that cannot 
be measured so easily. For example: 


w whether members are Satisfied with particular programs, 
policies or administrative procedures, 


= what members think about a community or group problem, 


. whether members are ready for a proposed change. 


Our most valuable source of evidence in evaluating community group activities 
is the members' freely expressed opinions. Techniques for getting this 
evidence can vary from informal remarks to elaborate surveys. What is most 
important is that the problem should be approached from every angle so that 
evidence from one source may be compared with the evidence from another. 
This approach increases the objectivity and soundness of the conclusion. 


There are many devices for getting to know the views and feelings of a group. 
One of the simplest is the buzz-group, a technique for involving the members 
in program planning. At each meeting, the members may form groups of six 
or eight without leaving their places. Each buzz-group chooses a spokesman 
and after about ten minutes discussion he will report the program suggestions 
of his group to the general meeting. It is the spokesman's responsibility to 
ensure that everyone in the small group has a chance to contribute. 


Many organizations use post-meeting reaction slips. At the end of the meeting 
or program to be evaluated, a copy is handed to each member so that he can 
indicate, without signing his name, his reactions to specific questions. Great 
care must be taken in framing the questions so that they will not be answered 
thoughtlessly or automatically. 


The answers should also form a useful basis for planning future activities. 
Often some form of rating scale is needed so members can register shades 

of opinion between "yes" and "no" or say how much they liked or disliked a 
program, plan or idea. Excellent-good-fair-poor is a rating scale of four 
parts. A scale to rate the reaction of a group may also be divided into five 
parts, or three or ten, for that matter. A reaction scale in five parts might 
be attached to a question such as "How satisfactory did you find this lecture ?" 


Completely 
satisfactory 


Unsatisfactory 


EVALUATION IN PROGRAM PLANNING 


Good programming is perhaps the most important single factor in the vitality 
of any community organization. Let's consider an example. 


Suppose the chairman of a program committee is a former public health nurse 
with an enthusiastic interest in preventive health care. She arranges a series 
of meetings — all lectures, on the topic in which she is interested. She invites 
a qualified speaker to lecture. She and her committee make suitable physical 
arrangements for the meetings. The committee gets out invitations and 
encloses pamphlets giving background information. 


The speaker is highly trained and speaks at great length. During the first 
talk the members are restless, At the second meeting there is a drop in 
attendance. For the third meeting, only a handful appear. The series is an 
obvious failure. 


Applying some of the principles of evaluation, the failure may be analyzed: 


# Evaluation of the plan would have shown that the method of 
presentation was out of line with the members way of thinking. 


mw Through co-operative evaluation at the planning stage, the 
members would have had the opportunity to express their opinions. 


e Continuous evaluation would have provided the warning signals, 
and suggested ways of saving the later meetings from failure. 


= Functional evaluation, at any stage, would have led to a change 
of the program content or presentation. 


Before deciding on their program plan, the committee might have asked 
members whether they would like a series of lectures on a single topic. If 
that had been acceptable, a list of topics could have been drawn up, with 
spaces for suggestions, and each member given an opportunity to rate his 


choices on the list. The topics selected would then have been in keeping with 
the members wishes. 


After the first lecture, a general post-meeting questionnaire like the one 
shown might have been offered to the members. 


ASSESSMENT OF PROGRAM 


1. Please indicate your opinion of this program 


(a) poor fair good excellent 
(b) Comments 


2. Please give your suggestions for improvement 


Committee convenors and executive officers should apply the principles of 
evaluation to their own work. Their starting pointing is the question, "What 
should we be doing?" When an executive group has a clear, honest answer to 
that question, it may begin to collect evidence to show where they are in 
relation to the goal. 


The effectiveness of an executive can be measured in rated answers to key 
questions: 


a What proportion of the organization's programs are 
initiated by members rather than officers ? 


« How many members maintain their interest at a high 
level and participate actively in the programs? 


They may use some of the methods already suggested to find out the members' 
opinions of their work. 


USE OF AN OBSERVER 


Committees and executive officers might also ask an experienced outside 
observer to report after watching them in action for some time. A person 
who is not involved in the processes of the organization can often see more 
clearly than the participants. 


Questions the observer would ask are: 


w Does the group understand its purpose ? 
a Is the group progressing toward its objectives? If not, why? 


a Is the group making the best possible use of all the resources and 
facilities available to it? They may "belong" to the community and 
others, but can be shared. 


= Does the leadership of the organization help all members to 
participate and to take increasing responsibility ? 


The observer's report should be presented frankly and objectively. It is not 
necessary to refer to people by name or to criticize individuals. The obser- 
ver's responsibility is to describe the group process as he saw it. The group 
itself can draw the conclusions about the weakness or strength of its own 
procedures. 
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